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Statement 


Office  of  Workers’  Compensation  Programs: 
Goals  and  Monitoring  Are  Needed  to  Further 
Improve  Customer  Communications 


Mr.  Chairman  and  Members  of  the  Subcommittee: 

Thank  you  for  the  opportunity  to  testify  on  the  Department  of  Labor’s 
Office  of  Workers’  Compensation  Programs  (OWCP).  You  asked  us  to 
review  how  OWCP  communicates  with  injured  federal  workers,  agencies 
who  employ  these  individuals,  and  medical  and  other  service  providers 
who  are  involved  in  their  treatment.  In  general,  our  review  focused  on  how 
well  OWCP’s  performance  management  system  was  used  to  respond  to 
claimants’  and  other  customers’  inquiries,  both  over  the  telephone  and  in 
written  correspondence.  This  statement  responds  to  your  request  by 
describing  how  we  conducted  our  review  and  what  we  found. 

To  evaluate  OWCP’s  system,  we  used  criteria  suggested  by  the  National 
Partnership  for  Reinventing  Government  (NPR). 1  NPR  performed  an 
extensive  study  of  high  performance  customer  service  organizations  in  the 
private  sector  and  their  best  practices  for  providing  telephone  service. 

NPR  stated  that  the  level  of  service  a  customer  receives  should  not  vary 
significantly  across  an  organization.  NPR  also  found  that  model  customer 
service  organizations  in  the  private  sector  generally  follow  three  consistent 
approaches  or  best  practices  for  telephone  communications.  First,  they  set 
challenging  goals  for  meeting  callers’  needs  for  timely  and  accurate 
information.  Second,  they  collect  credible  performance  data  to  measure 
progress  in  attaining  those  goals.  Third,  they  continuously  improve 
telephone  service  by  using  the  performance  data  and  results  of  periodic 
surveys  of  customers  and  stakeholders  to  determine  levels  of  satisfaction. 


Results  in  Brief 


While  OWCP  uses  these  model  organizations’  approaches  to  some  extent 
in  communicating  with  its  customers,  its  efforts  frequently  did  not  equal 
those  in  the  private  sector  that  NPR  identified  or  in  the  three  federal  and 
state  agencies  we  surveyed  that  have  won  awards  for  customer  service. 
We  found  that  OWCP 


•  provided  widely  varying  service  levels  across  its  district  offices  for  those 
attempting  to  reach  OWCP  representatives  by  phone; 

•  did  not  set  any  goals  for  some  important  areas  of  telephone 
communications,  and  the  goals  it  did  set  for  telephone  and  written 
communications  generally  tended  to  be  less  exacting  than  the  goals  NRP 
suggests  or  that  other  customer  service  operations  we  surveyed 
established  for  themselves; 


1  The  National  Performance  Review,  now  entitled  the  National  Partnership  for  Reinventing 
Government,  was  begun  in  1993  under  the  direction  of  Vice  President  Gore.  NPR  directed  federal 
agencies  to  build  a  customer  focus  into  their  operations  to  eliminate  unnecessary  bureaucracy, 
streamline  processes,  and  serve  the  public  more  cost  effectively. 
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•  did  not  often  collect  timely  or  credible  performance  data  to  gauge  progress 
in  attaining  its  goals;  and 

•  did  not  adequately  survey  ir\jured  workers,  medical  providers,  and  others 
to  determine  levels  of  satisfaction,  or  follow  many  other  practices  that 
NPR’s  model  organizations  use  to  improve  customer  service. 

At  the  conclusion  of  my  testimony,  I  will  discuss  ways  that  we  believe 
OWCP  can  improve  its  telephone  and  written  communications  with 
customers. 

To  get  an  indication  of  whether  OWCP  was  providing  consistent  customer 
service  regardless  of  where  iryured  workers  live,  we  placed  a  total  of  2,400 
telephone  calls  to  OWCP’s  12  district  offices  (200  per  office).  During  these 
calls,  we  attempted  to  obtain  information  that  an  injured  federal  worker 
might  be  calling  to  request  (e.g.,  status  of  compensation  payments  or 
medical  bill  inquiries).  For  each  call,  we  recorded  how  successful  the 
district  office  was  in  providing  this  information  (e.g.,  busy  signal,  no 
answer  after  1  minute,  reached  voice  mail  system,  obtained  consistent 
information  about  claimant  from  the  automated  system,  spoke  to  an 
OWCP  representative). 

To  determine  what  goals  OWCP  had  set  to  improve  customer  service  and 
how  the  performance  data  from  these  measures  are  used  to  improve  the 
program,  we  (1)  visited  OWCP  headquarters  in  Washington,  D.C.,  and 
interviewed  officials  and  obtained  documentation,  including  strategic  and 
annual  operational  plans;  (2)  visited  5  of  the  12  OWCP  district  offices 
(Chicago,  Dallas,  San  Francisco,  Seattle,  and  Washington,  D.C.)  and 
interviewed  district  directors  and  others  and  obtained  documentation  on 
office  practices;  and  (3)  surveyed  the  remaining  7  OWCP  district  offices  to 
collect  some  similar  information,  such  as  the  extent  to  which  telephone 
inquiries  are  recorded. 

To  compare  OWCP’s  goals  and  practices  for  telephone  communication 
with  those  of  model  organizations,  we  surveyed  three  agencies  that  have 
won  awards  for  their  telephone  communication  practices:  the  Social 
Security  Administration  (SSA),  Department  of  Veterans  Affairs’  Benefits 
Administration  (VBA),  and  state  of  Ohio’s  Bureau  of  Workers’ 
Compensation  (Ohio’s  BWC).  We  asked  OWCP  and  the  three  agencies  to 
identify  which  of  95  telephone  “best  practices”  they  use  that  NPR 
identified  in  its  1995  study. 

We  did  our  work  between  January  and  September  2000.  (See  appendix  I 
for  additional  information  regarding  scope  and  methodology.) 
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Background 


OWCP  is  responsible  for  adjudicating  and  administering  claims  of  work- 
related  injuries  and  illnesses  as  authorized  by  the  Federal  Employees’ 
Compensation  Act  (FECA)  (5  U.S.C.  8101  et  seq.,  as  amended).2  The  FECA 
program  covers  nearly  3  million  active  duty  civilian  federal  employees, 
providing  benefits  to  those  it  determines  sustain  an  injury  or  illness  in  the 
performance  of  duty  worldwide. 

During  fiscal  year  1999,  FECA’s  costs  totaled  about  $1.9  billion  in 
compensation,  medical,  and  death  benefits,  and  federal  employees  filed 
about  167,000  ir\jury  notices.  At  the  end  of  fiscal  year  1999,  OWCP  was 
administering  about  243,000  ongoing  figury  cases,  including  from  previous 
years,  for  partial  or  total  disability. 

According  to  OWCP  officials,  they  receive  an  estimated  2.6  million  phone 
calls  and  5.5  million  pieces  of  mail  each  year  from  customers-claimants, 
medical  providers,  agencies,  and  others.  Some  mail  requires  a  response — 
for  example,  congressional  inquiries  on  behalf  of  constituents.  However, 
district  office  officials  said  they  believed  that  most  of  the  mail  does  not 
require  a  response.  For  example,  medical  reports  are  used  to  assist  claims 
examiners  in  adjudicating  cases  but  do  not  usually  require  a  response. 
Although  OWCP  did  not  know  what  proportion  of  its  mail  requires  a 
response,  district  office  officials’  estimates  ranged  from  1  percent  to  7 
percent. 

The  telephone  calls  and  written  correspondence  are  handled  primarily  by 
12  OWCP  district  offices  nationwide,  which  had  a  total  of  about  900 
employees  as  of  December  1, 1999.  These  district  offices  operate  under  the 
authority  and  guidance  of  OWCP  headquarters  and  are  responsible  for 
adjudicating  claims  from  figured  workers,  approving  wage  loss  claims, 
paying  medical  bills,  and  responding  to  inquiries  from  customers.  Each 
district  office  is  responsible  for  providing  services  to  claimants  living  in 
several  states. 

OWCP  has  taken  actions  to  improve  customer  communications  over  the 
last  5  years.  These  actions  have  included 

•  implementing  automated  voice  response  systems  at  all  12  district  offices  to 
provide  information  on  the  claims,  such  as  the  status  of  bill  payments,  24 
hours  a  day; 


2  OWCP  also  adjudicates  and  administers  claims  authorized  by  the  Longshore  and  Harbor  Workers’ 
Compensation  Act,  which  covers  employees  engaged  in  maritime  employment,  and  for  recipients  of 
Black  Lung  compensation. 
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•  expanding  automated  voice  response  systems  to  allow  pharmacy  staff  to 
verify  claimant’s  eligibility  and  the  amounts  of  drug  payments  authorized; 

•  beginning  the  process  of  converting  incoming  medical  bills  and  other 
correspondence  to  a  computerized  format  to  make  the  information 
available  to  district  office  representatives  via  their  computer  terminals, 
enabling  them  to  answer  more  queries  during  initial  calls; 

•  giving  federal  agencies,  unions,  and  congressional  staff  direct  computer 
access  to  information  they  need  to  deal  with  their  employees’  or 
constituents’  cases;  and 

•  initiating  a  communications  redesign  project  last  year-which  included 
establishing  a  redesign  team  comprised  of  union  members  and 
management  to  propose  standards,  reengineer  practices,  and  make  other 
improvements  in  OWCP’s  communications-and  investigating  best 
practices  in  public  and  private  organizations. 

OWCP  has  also  taken  actions  when  its  monitoring  systems  have  indicated 
that  district  offices  have  failed  to  meet  goals  for  responsiveness  to 
telephone  inquiries.  For  example,  when  district  offices  failed  to  meet  goals 
for  responding  to  telephone  inquiries  for  one  or  more  quarters  of  a  fiscal 
year,  OWCP’s  national  office  counseled  the  district  directors  and  required 
plans  for  improvement. 

In  addition,  OWCP’s  budget  request  for  fiscal  year  2001  requested  funding 
for  a  toll  free  800  telephone  number  for  medical  authorizations,  for 
telephone  system  hardware  upgrades,  for  additional  communication 
specialists,  and  for  expanded  access  to  automated  information  for  injured 
workers.  As  of  September  22,  2000,  the  House  and  Senate  appropriation 
committees  for  OWCP  had  decided  not  to  fund  this  request. 


Telephone  Customer 
Service  Levels  Varied 
Widely  Across  District 
Offices 


Although  NPR  has  stated  that  the  level  of  service  a  customer  receives 
should  not  vary  significantly  across  an  organization,  we  found  that  service 
levels  varied  widely  for  those  attempting  to  reach  OWCP  representatives 
by  phone.  As  figure  1  shows,  the  extent  to  which  we  were  unable  to  access 
district  offices’  telephone  systems  on  our  2,400  calls — t  hat  is,  where  there 
was  a  busy  signal,  no  answer  after  1  minute,  or  a  message  erroneously 
stating  that  the  phone  number  was  invalid — ranged  from  0  percent  in 
Boston  to  54  percent  in  Jacksonville. 
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60 


[  [  Percentage  of  calls  failing  to  access  an  OWCP  telephone  systems 

Source:  GAO  analysis  of  telephone  survey  calls  to  district  offices. 


The  reasons  given  for  our  not  being  able  to  access  the  phone  system 
varied.  For  example,  in  Jacksonville  we  frequently  experienced  busy 
signals  because  district  officials  there  said  that  they  believe  it  is  better  for 
a  customer  to  receive  a  busy  signal  than  to  remain  on  hold  for  an  extended 
period  of  time  at  the  caller’s  expense.  Conversely,  for  San  Francisco,  we 
experienced  a  relatively  high  frequency  of  calls  with  no  answer  after  15 
rings  (about  1  minute).  The  San  Francisco  district  director  told  us  this  was 
caused  by  a  flaw  in  the  phone  system  that  has  existed  for  years.  She  said 
that,  although  the  customer  hears  the  phone  ringing,  the  system  does  not 
recognize  that  someone  is  calling.  She  also  said  that  she  had  spoken  with 
officials  from  the  phone  company  as  well  as  communication  officials  in  the 
Department  of  Labor,  but  that  the  problem  remained  unresolved.  OWCP’s 
Acting  Director  said  that  the  problem  with  the  phone  system  had  been 
resolved  as  of  September  13,  2000.  The  Acting  Director  also  stated  that  the 
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Washington,  D.C.,  office  had  purchased  an  additional  eight  telephone  lines 
in  late  July  2000  and  that  he  believes  this  will  increase  the  system’s 
accessibility. 

We  also  found  that  our  ability  to  speak  to  an  OWCP  employee  varied 
significantly  across  districts.  Of  the  2,400  calls  we  made,  1,200  calls  were 
to  either  an  office  phone  number  designated  for  contacting  an  employee  or 
a  central  phone  number  that  gives  callers  an  option  for  contacting  a 
representative.  As  figure  2  shows,  the  rates  at  which  we  were  unable  to 
reach  any  employee  within  5  minutes  ranged  from  13  percent  to  97  percent 
of  the  calls.3  In  three  offices — Jacksonville,  Dallas,  and  New  York — we 
were  unable  to  access  an  employee  on  97, 86,  and  80  percent  of  the  calls, 
respectively.4 


3  We  chose  to  wait  no  longer  than  5  minutes  for  a  representative  from  the  time  we  made  a  selection 
requesting  a  representative  because  we  believed  this  was  a  conservative  approach,  given  VBA’s  goal  of 
reaching  a  representative  within  3  minutes  and  NPR’s  guidance  and  Ohio  BWC’s  goal  of  30  seconds. 

4  We  also  placed  1,200  calls  initially  to  the  district  offices’  automated  telephone  voice  response 
systems,  followed  by  an  attempt  to  contact  a  representative.  The  proportion  of  representatives  we 
were  unable  to  contact  using  this  approach  was  even  higher  than  when  using  our  other  approach. 
However,  we  made  a  conservative  assumption  that  a  caller  whose  primary  reason  for  calling  was  to 
speak  to  a  representative,  would  call  the  specific  number  for  accessing  a  representative  at  those 
district  offices  with  such  a  number,  or  immediately  attempt  to  access  a  representative  at  an  office  with 
only  one  number. 
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Access  an  OWCP  Representative  by 
District  Offices 
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Percentage 

100 


[  |  Percentage  of  calls  failing  to  access  a  representative 

Source:  GAO  analysis  of  telephone  survey  calls  to  district  offices. 


The  most  frequent  reasons  why  our  telephone  calls  did  not  reach  an 
employee  were  that  we  were 

still  receiving  a  busy  signal  or  no  answer  after  15  rings, 

transferred  to  a  voice  mail  box  after  selecting  the  option  to  speak  to  a 

representative, 

still  on  hold  5  minutes  after  selecting  an  option  to  speak  to  an  employee,  or 
disconnected  after  selecting  an  option  to  speak  to  an  employee. 

Officials  at  the  five  district  offices  we  visited  said  that  there  were  too  few 
employees  to  both  answer  the  phones  and  adjudicate  claims  as  well  as 
perform  the  other  services  that  they  must  provide.  On  the  other  hand,  an 
official  at  OWCP  headquarters  said  that  they  did  not  want  to  reward  those 
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offices  with  the  lowest  telephone  access  rates  by  giving  them  part  of 
another  office’s  staff  allocation. 

When  we  made  our  2,400  telephone  calls,  we  also  attempted  on  1,200  of 
those  calls  to  compare  the  information  on  actual  injured  workers’  claims 
provided  to  us  by  OWCP  headquarters  officials  with  the  same  information 
available  on  that  claimant  through  district  offices’  telephone  systems.5  For 
example,  if  OWCP  headquarters  told  us  that  claimant  Mary  Smith  was 
mailed  a  compensation  check  of  $550,  would  the  district  office  where 
Mary’s  claim  was  handled  provide  us  with  this  same  information? 

We  did  not  include  604  of  the  1,200  calls  in  our  analysis:  138  calls  where  we 
could  not  access  the  phone  system  for  various  reasons  (e.g.,  busy  signal); 
43  calls  where  we  could  access  the  phone  system  but  not  the  interactive 
voice  response  system  for  various  reasons  (e.g.,  claim  number  was 
different  from  that  provided  by  headquarters),  and  423  calls  where  we 
could  not  compare  the  information  because  it  had  been  updated  after 
OWCP  headquarters  provided  it  to  us. 

For  the  remaining  596  calls,  the  extent  to  which  district  offices  provided  us 
with  consistent  claims  information  ranged  from  88  percent  to  100  percent. 
(See  appendix  II  for  information  on  the  accuracy  of  each  district  office’s 
interactive  voice  system.)  Most  of  the  inconsistent  information  involved 
the  dates  and  amounts  of  claimants’  compensation  checks. 

Other  communication  practices  also  varied  significantly  across  district 
offices: 

•  The  Dallas  office,  unlike  most  others,  used  e-mail  for  medical 
authorizations,  congressional  contacts,  and  general  inquiries.  The  four 
other  district  offices  we  visited  did  not  use  e-mail  because  of  Privacy  Act 
concerns. 

•  The  national  office  and  four  district  offices  have  taken  steps  to  provide 
customers  information  through  the  Internet,  while  others  have  not.  The 
Internet-linked  offices  have  established  a  World  Wide  Web  page  to  provide 
information  about  the  workers’  compensation  program  and  the  district 
offices’  procedures  and  practices. 


5  For  the  1,200  calls  we  placed  attempting  to  obtain  automated  voice  information,  we  were  able  to 
access  most  of  the  district  offices’  telephone  systems  on  a  higher  proportion  of  calls  than  we  did  for 
the  1,200  calls  attempting  to  access  a  representative.  For  calls  attempting  to  access  the  automated 
system,  we  were  unable  to  access  district  offices’  telephone  systems  on  from  0  to  37  percent  of  the 
calls  to  each  office. 
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•  Most  district  offices  had  representatives  available  to  answer  the  phone  7  or 
more  hours  per  day,  but  three  offices — New  York,  Philadelphia,  and 
Boston-were  available  by  phone  6  hours  or  less  per  day,  and  one  of  these 
offices — Boston — had  representatives  available  for  only  4.5  hours. 


OWCP  Did  Not  Have 
Goals  for  Some 
Important  Customer 
Service  Areas 


OWCP  had  not  set  any  goals  in  some  important  areas  of  telephone 
communications  and  the  goals  it  did  set  for  telephone  and  written 
communications  allowed  OWCP  more  time  to  provide  responses  to 
customers  than  NPR  suggests  for  telephone  communications  or  that  other 
organizations  we  surveyed  allowed.  For  example,  NPR  suggests  the 
following  telephone  service  goals: 


•  99  percent  of  callers  access  the  telephone  system; 

•  98  percent  of  callers  reach  a  customer  service  representative,  and  the  time 
waiting  on  line  be  no  more  than  30  seconds;  and 

•  85  percent  of  callers’  inquiries  should  be  resolved  during  the  first  call. 

These  three  basic  goals  focus  on  meeting  callers’  needs  for  timely  and 
accurate  information.  The  other  agencies  we  contacted-SSA,  VBA,  and 
Ohio’s  BWC-varied  in  whether  they  established  goals  for  these  measures. 
Three  had  goals  for  telephone  access,  two  had  goals  for  the  portion  of 
callers  reaching  representatives  and  the  time  they  have  to  wait  on  line,  and 
one  had  a  goal  for  resolving  inquiries  on  the  first  call. 

OWCP  had  not  set  goals  that  conform  to  any  of  these  three  goals.  OWCP 
did  have  a  goal  to  return  90  percent  of  phone  calls  not  related  to  medical 
authorizations  within  3  days  to  persons  who  leave  messages.  That  goal 
could  be  met  by  OWCP’s  calling  the  people  within  3  days,  giving  them  the 
status  of  their  claims,  and  saying  that  the  answer  to  their  questions  would 
follow  at  a  later  time.  However,  OWCP’s  Acting  Director  noted  that  this  is 
the  only  response  possible  for  many  calls  when  OWCP  lacks  information, 
such  as  doctors’  reports,  needed  to  resolve  the  caller  inquiry.  OWCP  also 
had  a  separate  goal  established  in  fiscal  year  1999  to  return  95  percent  of 
calls  related  to  medical  authorizations  within  3  days. 


NPR  estimates  that  organizations  that  answer  a  caller’s  question  on  the 
first  call  will  spend  less  time  and  about  half  the  resources  as  organizations 
that  take  multiple  calls  to  answer  inquiries.  OWCP’s  Acting  Director  said 
that  he  recognized  the  benefits  of  answering  the  calls  the  first  time  and 
that  an  ongoing  OWCP  program  to  make  more  claimant  information 
available  on  district  office  computer  terminals  could  help  achieve  such  a 
goal.  He  added,  however,  that  he  believed  establishing  a  goal  for  answering 
queries  would  be  more  appropriate  for  an  organization  with  a  call  center 
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whose  employees’  only  responsibility  is  answering  telephone  calls.  He 
explained  that  because  district  offices  have  many  other  responsibilities  in 
addition  to  answering  calls,  some  district  offices  prefer  to  direct  most  calls 
to  voice  mail  and  respond  at  a  later  time.  Several  district  directors  told  us 
that,  if  there  were  such  a  goal,  assigning  additional  employees  to  answer 
calls  would  take  time  away  from  their  adjudication  of  claims. 

The  scope  of  NPR’s  study  did  not  include  identifying  what  goals  the  private 
sector  has  for  responding  to  written  inquiries,  as  it  did  for  telephone 
communications.  Thus,  we  could  not  compare  the  goals  that  OWCP  has 
established  for  the  timeliness  of  written  communications  with  an  NPR 
suggested  standard. 

Nevertheless,  for  nonpriority  mail  requiring  a  response,  OWCP  had  a  goal 
of  responding  to  85  percent  within  30  days.  OWCP  also  had  goals  for 
responding  to  priority  mail  from  Congress:  90  percent  within  14  days  and 
98  percent  within  30  days.  These  goals  do  not  compare  favorably  to  VBA’s 
goal  of  responding  to  all  written  benefit  inquiries  within  10  workdays  and 
Ohio’s  BWC’s  goal  of  responding  to  written  requests  the  same  day,  or 
within  24  hours  of  the  request’s  being  referred  to  another  section  of  the 
Bureau.6 

OWCP  also  did  not  have  a  national  goal  for  responding  to  requests  for 
medical  authorizations  in  the  mail,  through  e-mail,  or  by  fax.  Nonetheless, 
the  five  district  offices  we  visited  gave  written  medical  authorization 
requests  received  by  mail  the  same  priority  that  OWCP  gave  congressional 
correspondence.  Several  of  these  five  offices  have  also  established  their 
own  goals  for  medical  authorizations  received  by  e-mail  or  fax.  For 
example,  the  Dallas  district  office  encourages  claimants  to  use  e-mail  for 
medical  authorizations.  Dallas  had  a  goal  of  responding  to  90  percent  of  e- 
mails  within  24  hours.  The  Chicago  district  office  received  95  percent  of  its 
medical  authorization  requests  by  telephone.  The  district  office  has  chosen 
to  use  the  goal  for  medical  authorizations  received  by  phone — 95  percent 
within  3  days — for  authorization  requests  received  by  fax. 


6  Prior  GAO  testimony  (Veterans  Benefits  Administration:  Problems  and  Challenges  Facing  Disability 
Claims  Processing.  GAO/T-HEHS/AIMD-OO-146)  stated  that  VBA— with  an  average  time  for  processing 
claims  of  205  days-was  far  from  reaching  its  strategic  goal  of  74  days.  While  VBA  did  not  meet  its  goal 
for  processing  claims,  VBA  officials  provided  reports  stating  that  it  did  meet  the  10-day  goal  for 
responding  to  nonpriority  correspondence  97  percent  of  time  in  fiscal  year  1999.  A  VBA  official  said 
that  just  telling  inquirers  that  they  would  be  contacted  at  a  later  date  with  an  answer  did  not  constitute 
a  valid  response  for  purposes  of  calculating  the  rate.  We  did  not  independently  verify  VBA’s  response 
rates.  SSA  does  not  have  goals  for  responding  to  written  correspondence. 


Page  10 


GAO-01-72T 


Statement 

Office  of  Workers’  Compensation  Programs:  Goals  and  Monitoring  Are  Needed  to  Further 
Improve  Customer  Communications 


OWCP  Lacked  Valid 
Information  to 
Measure  Progress  in 
Achieving  Goals 


Often,  OWCP  did  not  collect  credible  performance  data  to  gauge  progress 
in  attaining  its  goals.  Credible  performance  information  is  essential  for 
accurately  assessing  agencies’  progress  toward  meeting  existing  goals  and 
for  setting  new  goals.7  Decisionmakers  must  have  assurance  that  the 
program  and  financial  data  being  used  will  produce  complete,  credible, 
useful,  and  consistent  data  in  a  timely  manner  if  these  data  are  to  inform 
decisionmaking. 


Progress  Measures  for  OWCP’s  system  for  measuring  its  goal  of  3  days  for  returning  phone  calls 

rr„imll„np  Timolinocc  Woro  to  those  who  left  a  messaSe  that  required  a  response  did  not  yield  valid 

“  e  e  timeliness  measurements.  It  could  do  this  by  either  creating  a  record  of  all 

Invalid  such  calls  or  of  a  statistically  valid  sample.  While  all  calls  may  not  require  a 

response,  district  office  officials  have  stated  that  most  callers  have 
inquiries  that  require  a  response.  The  national  office  suggested-but  did  not 
require-that  district  employees  use  a  standardized  computer  program  (CA- 
110)  to  make  a  recording  of  all  calls  requiring  a  response,  as  well  as  of 
those  in  which  the  content  is  relevant  to  adjudicating  decisions.  OWCP  did 
not  require  the  use  of  this  program  for  all  calls  because  some  district  office 
employees  have  complained  about  taking  time  away  from  their  other  tasks 
to  record  the  information,  such  as  the  date  and  nature  of  the  call. 

We  found  that  all  12  district  offices  used  the  CA-110  system  to  some 
extent.  However,  two  of  the  five  offices  we  visited — Dallas  and  Seattle- 
told  us  that  they  entered  only  about  15  percent  or  fewer  of  all  calls 
requiring  a  response  and  did  not  enter  calls  in  the  systematic  manner  that 
would  be  necessary  to  yield  valid  results.  The  other  three  offices — 
Chicago,  San  Francisco,  and  Washington,  D.C.-estimated  that  they  entered 
about  75,  75,  and  95  percent  of  the  calls  requiring  a  response,  respectively. 
However,  our  analysis  of  the  number  of  calls  received  and  the  number  of 
calls  recorded  in  the  CA-110  system  suggests  that  these  estimates  are  high.8 
For  example,  San  Francisco  estimated  that  it  entered  75  percent,  but  for  a 
3-month  period  in  fiscal  year  2000,  San  Francisco  received  76,238  calls  and 
entered  14,502,  or  19  percent,  in  the  CA-110  system. 

Moreover,  four  offices — Cleveland,  San  Francisco,  Seattle,  and 
Washington,  D.C-also  used  alternative  methods  to  record  and  track  a 
portion  of  their  calls.  Each  of  these  offices  was  supposed  to  follow  a 


7  Managing  for  Results:  Challenges  Agencies  Face  in  Producing  Credible  Performance  Information 
(GGD-00-52,  Feb.  4,  2000). 

8  The  five  district  offices  provided  us  with  information  on  the  number  of  calls  received  that  was  based 
on  periods  from  4  weeks  to  3  months.  These  offices  also  provided  us  with  the  number  of  calls  recorded 
in  their  CA-110  systems  over  various  periods  of  time. 
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sampling  plan  approved  by  OWCP  headquarters  when  recording 
information.  However,  the  OWCP  Acting  Director  said  that  each  of  these 
four  offices  had  developed  a  modified  version  of  the  sampling  plan  and 
that  their  plans-while  approved  by  OWCP-were  probably  not  statistically 
valid.  We  reviewed  the  national  office’s  sampling  plan  for  the  data  to  be 
entered  into  these  systems  and  also  believe  that  this  plan  would  not  yield 
statistically  valid  results  even  if  implemented  as  designed. 

The  telephone  response  rates  developed  using  the  CA-110  and  other 
systems  indicated  that  the  district  offices  were  generally  meeting  their 
timeliness  goals.  However,  because  the  performance  data  were  not 
statistically  valid  or  collected  in  enough  cases,  OWCP  could  not  determine 
whether  these  goals  were  being  met. 


Progress  Measures  for 
Timeliness  of  Written 
Correspondence  Were  Also 
Invalid 


We  also  found  problems  in  the  methods  that  OWCP  used  to  measure 
timeliness  goals  for  written  correspondence  that  undermine  the  usefulness 
of  the  data.  Each  district  office  was  required  to  take  a  statistical  sample  of 
incoming  general,  nonpriority  correspondence,  and  then  record  and  track 
the  correspondence  to  determine  whether  it  was  responded  to  within  30 
days.  The  results  of  these  samples  were  to  be  reported  to  the  national 
office  on  a  quarterly  basis.  Foot  of  the  five  district  offices  we  visited  each 
had  a  different  approach  for  sampling  such  correspondence,  and  each 
stated  that  its  approach  was  not  scientifically  developed  or  developed  in  a 
manner  that  would  produce  valid  results  if  projected  to  the  universe  of  all 
responses.9  For  example,  the  Dallas  district  office  required  each  claims 
examiner  to  provide  his  or  her  supervisor  four  pieces  of  written 
correspondence  requiring  a  response  per  month  to  determine  whether  the 
response  was  provided  within  30  days.  We  are  concerned  about  the 
validity  of  this  approach  because,  among  other  things,  the  potential  exists 
for  the  claims  examiner  to  give  the  supervisor  only  those  letters  to  which 
the  response  was  timely. 


The  Seattle  district  office  required  claims  examiners  to  log  in  all  general 
correspondence  received  every  Wednesday  that  required  a  response. 
Supervisors  were  to  review  the  claimants’  files  for  these  letters  after  30 
days  to  determine  whether  a  response  had  been  sent  within  the  30-day 
goal. 


We  are  concerned  about  the  validity  of  this  approach  because,  among 
other  things,  OWCP  employees  may  devote  extra  attention  to  responding 
to  letters  arriving  on  Wednesday  at  the  expense  of  letters  arriving  on  other 


9  Chicago  district  office  officials  stated  that  they  track  all  of  their  nonpriority  written  correspondence. 
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days.  Although  the  performance  data  showed  that  this  timeliness  goal  for 
fiscal  year  1999  was  generally  met  by  all  district  offices,  the  results  are 
probably  not  statistically  reliable. 

Conversely,  OWCP  did  seem  to  have  a  valid  and  reliable  system  for 
tracking  responses  to  priority  mail  involving  congressional  requests.  For 
example,  the  date  of  receipt  of  all  congressional  correspondence  was  to  be 
recorded  in  the  Priority  Correspondence  Tracking  System.  This  system 
provides  reports  that  track  each  piece  of  correspondence  until  a  response 
is  provided. 


As  I  said  earlier,  the  third  approach  that  NPR  found  model  organizations 
followed  was  to  continuously  improve  customer  service  by  using 
performance  data — including  surveys  of  important  customers  and 
stakeholders-  to  identify  how  and  where  improvements  are  needed.  We 
found  that  OWCP’s  efforts  in  this  area  fell  well  short  of  the  best  practices 
that  NPR  found  in  the  private  sector  and  in  the  three  agencies  we 
surveyed. 

Let  me  first  state  the  obvious-OWCP  did  not  measure  progress  toward 
goals  that  it  did  not  establish  in  the  first  place.  OWCP  did  not  have  goals — 
nor  did  it  have  related  measures — for  three  basic  areas  of  telephone 
communications: 

•  the  percentage  of  callers  able  to  access  the  telephone  system, 

•  the  percentage  of  callers  who  can  reach  a  customer  service  representative 
and  their  time  waiting  on  line,  and 

•  the  percentage  of  callers’  inquiries  that  are  resolved  during  the  first  call. 

Officials  from  SSA  and  VBA  told  us  that  setting  goals  for  those  areas  (e.g., 
telephone  access)  for  their  agencies  and  measuring  the  results  has  proven 
useful  in  identifying  areas  where  customer  satisfaction  levels  needed 
improving.  For  example,  data  from  SSA’s  customer  satisfaction  surveys  in 
1993  showed  that  access  was  the  single  biggest  factor  affecting  customer 
satisfaction.  According  to  an  SSA  official,  SSA  began  collecting  access  data 
and  round  that  callers  attempting  to  reach  SSA  at  the  busiest  times  were 
getting  busy  signals  50  percent  of  the  time.  SSA  established  a  telephone 
access  goal  and  continued  to  collect  access  data,  explore  new 
technologies,  and  acquire  additional  telephone  capacity.  By  1996,  SSA  said, 
it  was  able  to  set  and  achieve  a  goal  of  95  percent  of  the  callers  reaching 
the  system  within  5  minutes,  and  customer  satisfaction  scores  improved 
accordingly. 


OWCP  Often  Lacked 
Customer  Satisfaction 
Data  to  Know  How  and 
Where  Improvements 
Were  Needed 
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My  point  in  using  this  example  is  that  without  goals  or  measures  in  these 
three  basic  high  priority  areas,  OWCP  was  not  in  a  position  to  know  what 
levels  of  customer  service  it  was  providing  and  where  and  how  telephone 
services  needed  to  be  improved. 

For  the  goals  that  OWCP  did  establish,  we  found  that  it  often  did  not 
collect  credible  performance  data  from  surveys  of  (1)  injured  workers,  (2) 
medical  providers,  or  (3)  employees  that  was  sufficiently  reliable  or  done 
in  a  timely  manner  to  measure  progress  or  to  set  goals  for  improving 
customer  service. 


OWCP  Did  Little  to  Gauge 
Customer  Satisfaction 
Levels 


NPR  found  that  model  telephone  service  organizations  in  the  private 
sector  survey  (1)  their  customers  frequently  to  determine  how  satisfied 
they  are  with  the  services  provided  and  (2)  their  employees  who  are 
answering  the  phones  for  job  satisfaction  levels  and  ideas  to  improve  their 
services.  In  addition,  Executive  Order  12862,  issued  September  11, 1993, 
directs  departments  and  agencies  to  survey  customers  to  determine  the 
kind  and  quality  of  services  they  want  and  their  level  of  satisfaction  with 
existing  services. 


Every  organization  must  decide  how  frequently  it  can  survey  its  customers 
in  a  cost-effective  manner.  However,  we  found  that  OWCP  (1)  did  not 
obtain  information  on  injured  workers’  satisfaction  with  services  as 
frequently  and  by  utilizing  as  many  techniques  as  other  model 
organizations  do;  (2)  did  not  survey  other  important  stakeholders,  such  as 
medical  providers  and  federal  agencies;  and  (3)  did  not  survey  the  OWCP 
employees  who  are  answering  the  phones. 

OWCP  did  do  customer  satisfaction  surveys  of  injured  workers.  Since 
1996,  OWCP  has  hired  a  contractor  to  conduct  customer  satisfaction 
surveys  about  once  each  year  to  determine  claimants’  perceptions  of 
several  aspects  of  the  FECA  program,  including  overall  service,  the 
timeliness  of  responses  to  telephone  inquiries,  and  the  timeliness, 
thoroughness,  and  accuracy  of  written  responses  to  claimants’  inquiries.10 
The  claimants  are  selected  on  a  random  sample  basis.  During  these 
surveys,  injured  workers  are  asked  to  recall  situations  that  occurred  up  to 
1  year  in  the  past. 


NPR  recommends  as  a  best  practice  that  organizations  survey  their 
customers  constantly  to  determine  satisfaction  levels  for  existing  services 


10  The  surveys  were  conducted  in  1996  and  1998  by  the  same  contractor.  In  1997,  OWCP  contracted 
with  another  contractor  to  perform  a  similar  survey. 


Page  14 


GAO-01-72T 


Statement 

Office  of  Workers’  Compensation  Programs:  Goals  and  Monitoring  Are  Needed  to  Further 
Improve  Customer  Communications 


and  to  gather  requests  for  new  services.  Ohio’s  BWC  surveys 
approximately  600  injured  workers  weekly  by  mail. 

The  OWCP  Acting  Director  said  that  OWCP  does  not  conduct  a  customer 
satisfaction  survey  of  medical  providers  because  it  had  difficulty  doing  so 
in  the  past.  That  is,  OWCP  officials  said  they  had  previously  attempted  to 
survey  medical  providers;  however,  when  they  called  the  representative  in 
the  medical  provider’s  office  often  could  not  identify  the  individual  who 
had  previously  called  OWCP. 

Some  district  office  directors  and  other  officials  cited  outreach  programs 
that,  while  not  necessarily  systematic,  were  an  attempt  to  gain  more  input 
from  a  broader  selection  of  customers.  For  example,  the  Seattle  district 
office,  in  May  2000,  initiated  a  program  to  begin  calling  a  sample  of  all 
telephone  callers  within  the  same  week  they  called.  The  district  director 
said  that  the  office  asked  the  callers  whether  the  response  was  appropriate 
and  if  the  caller  was  satisfied  with  the  representative’s  response.  OWCP 
national  office  officials  said  that  these  efforts  did  not  ensure  that 
consistent  questions  were  used  or  that  a  random  sample  of  all  district 
office  customers  was  surveyed. 

OWCP  also  has  not  surveyed  its  own  employees  regarding  customer 
service  or  employee  satisfaction  within  the  last  5  years.  According  to  NPR, 
employee  satisfaction  is  measured  as  routinely  as  customer  satisfaction  in 
model  customer  service  organizations.  NPR  cited  as  benefits  of  such 
employee  surveys:  obtaining  information  on  how  to  improve  the  work 
processes  that  lead  to  improved  customer  service,  as  well  as  identifying 
employee  morale  issues  that  could  lead  to  customer  service  problems.  In 
addition,  Executive  Order  12862  directed  agencies  to  survey  front-line 
employees  on  barriers  to,  and  ideas  for,  matching  the  best  in  business. 
Officials  from  Ohio’s  BWC  told  us  that  they  have  systems  in  place,  such  as 
an  annual  employee  survey  that  captures  information  about  employee 
morale. 


OWCP  Followed  Fewer 
NPR  Best  Practices  Than 
the  Other  Three  Agencies 
We  Visited 


NPR  identified  95  best  telephone  service  practices  of  customer  service 
organizations  in  the  private  sector.  Of  these  95,  we  categorized  20 
practices  as  being  related  to  performance  measurement  and  68  as  related 
to  improving  customer  service  through  use  of  performance  data.11  Each  of 
these  “practices”  also  contained  several  major  activities.  For  example,  the 
practice  of  managing  customer  dissatisfaction  with  telephone  service 


11  We  determined  that  the  other  seven  best  practice  categories  were  not  related  to  either  improving 
customer  service  through  the  use  of  performance  and  other  data  or  to  measuring  performance. 
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included  related  activities,  such  as  establishing  a  dedicated  customer 
relations  team  to  receive  complex  customer  issues  and  complaints  and 
recording  caller  complaints  for  use  in  identifying  the  root  causes  of 
problems.  While  no  organization  would  be  expected  to  apply  all  of  these 
best  practices,  we  wanted  to  determine  the  extent  to  which  OWCP  was 
applying  these  practices  in  comparison  to  SSA,  VBA,  and  Ohio’s  BWC. 
Consequently,  we  asked  these  organizations  to  characterize  to  what 
extent — “all,”  “some,”  or  “none”-they  followed  the  activities  within  each  of 
these  practices. 

Of  the  20  best  practices  related  to  measuring  performance,  OWCP  stated 
that  two  best  practices  were  not  applicable  because  they  applied  to  call 
center  operations  and  OWCP  does  not  have  any  call  centers.  Of  the  18  that 
OWCP  said  were  applicable,  it  reported  that  it  performed  “all”  or  “some”  of 
9  of  the  18  practices,  or  50  percent.  This  compares  to  SSA,  VBA,  and  Ohio’s 
BWC,  who  reported  that  they  performed  20, 17,  and  20,  or  100,  85,  and  100 
percent,  of  all  20  NPR  best  practices,  respectively.  Of  the  9  practices 
OWCP  indicated  it  did  not  perform,  SSA,  VBA,  and  Ohio’s  BWC  responded 
that  they  applied  8  of  them  to  all  or  some  extent.  Examples  of  these  8 
performance  measuring  practices  follow. 

•  Call  monitoring:  Senior  managers  regularly  listen  in  on  live  calls  in  order  to 
stay  in  touch  with  the  customer.  Team  leaders  participate  in  group 
monitoring  sessions  to  ensure  consistency  of  measurement. 

•  Accessible  to  customers:  A  customer  feedback  loop  is  built  into  every 
phase  of  the  customer  service  delivery  process.  It  is  convenient  and  easy 
for  customers  to  contact  world-class  organizations. 

Of  the  68  best  practices  for  improving  customer  service,  OWCP  said  that  6 
practices  were  not  applicable  because  they  applied  to  call  centers.  Of  the 
62  practices  OWCP  said  were  applicable,  OWCP  reported  that  they 
generally  followed  to  all  or  some  extent  31  of  the  62  practices,  or  50 
percent.  This  compares  to  SSA,  VBA,  and  Ohio’s  BWC,  who  reported  that 
they  followed  65,  51,  and  66,  or  96,  75,  and  97  percent,  of  the  68  NPR  best 
practices  to  all  or  some  extent,  respectively.  Of  the  31  practices  OWCP 
indicated  it  did  not  perform,  SSA,  VBA,  and  Ohio’s  BWC  each  responded 
that  they  applied  18  of  them  to  all  or  some  extent.  Examples  of  these  18 
practices  follow. 

•  Information  queuing:  Callers  waiting  in  the  queue  are  provided  with 
information  as  to  the  expected  length  of  delay,  allowing  them  to  choose 
whether  to  stay  in  queue  or  hang  up. 
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•  Resource  allocation  strategies:  Continuous  evaluations  of  key  performance 
indicators  help  to  ensure  the  appropriate  alignment  of  resource  allocations 
with  planning  objectives.  Well-established  benchmarking  programs  help 
identify  improvement  opportunities  for  cross-functional  teams.12 


Conclusions 


OWCP  has  been  concerned  about  the  level  of  services  that  it  provides  to  its 
customers — injured  workers,  medical  service  providers,  and  agencies.  The 
annual  surveys  of  injured  workers  that  OWCP  has  contracted  for  have 
surfaced  issues,  like  access  to  service  representatives,  that  OWCP  has 
taken  actions  to  address.  For  instance,  the  installation  of  the  automated 
response  systems  in  district  offices  have  given  customers  an  alternative 
means  of  getting  answers  to  common  questions.  OWCP  has  also  begun  to 
set  goals  and  use  data  about  goal  achievement  to  manage  the  customer 
service  aspect  of  district  office  operations. 

However,  OWCP’s  customer  service  efforts  fell  short  of  practices  used  in 
three  organizations  we  surveyed  and  in  model  organizations  in  the  private 
sector  that  NPR  studied.  According  to  our  telephone  survey,  customers’ 
access  to  OWCP’s  telephone  systems  and  to  customer  service 
representatives  varied  widely  by  district  office.  OWCP  had  not  established 
goals  for  several  important  aspects  of  telephone  service.  Where  OWCP  had 
set  telephone  or  written  communication  goals,  they  generally  tended  to  be 
less  exacting  than  the  goals  NPR  suggests  or  that  other  customer  service 
operations  we  surveyed  established  for  themselves.  Further,  for  those 
goals  OWCP  had  established,  the  systems  for  collecting  performance 
information  were  not  yielding  credible  information  that  OWCP  could  use 
to  make  properly  informed  decisions  about  its  customer  service 
operations.  In  addition,  OWCP  officials  reported  using  proportionately 
about  one  half  as  many  best  practices  related  to  measuring  performance 
and  proportionately  from  one  half  to  two  thirds  as  many  best  practices 
related  to  improving  customer  service  through  the  use  of  performance  and 
other  data  as  did  the  three  award  winning  customer  service  operations  we 
surveyed.  By  developing  goals  for  important  areas  of  telephone 
communications,  determining  if  the  timeliness  goals  for  telephone  and 


12  In  a  second  response  to  our  request  after  reviewing  our  draft  testimony,  OWCP  revised  its  responses 
to  15  of  the  best  practices,  including  increasing  from  8  to  14  the  number  of  practices  it  considered  not 
applicable  because  the  activities  applied  to  call  centers.  Based  on  OWCP’s  revised  response,  OWCP 
considers  3  of  the  20  best  practices  for  performance  measurement  not  to  be  applicable  and  reported 
that  it  performs  all  or  some  of  1 1  of  the  remaining  17  practices,  or  65  percent.  Of  the  68  best  practices 
for  improving  customer  service,  OWCP  said  that  11  were  not  applicable.  OWCP  reported  that  they 
followed  to  all  or  some  extent  39  of  the  remaining  57  practices,  or  68  percent.  We  reported  OWCP’s 
original  responses  in  comparison  to  those  of  the  other  organizations’  responses  because  the  other 
organizations  did  not  have  an  opportunity  to  revise  their  responses  to  our  survey.  In  addition,  we  did 
not  have  an  opportunity  to  analyze  OWCP’s  revisions,  including  its  position  on  the  NPR  practices  that 
it  viewed  as  applicable  only  to  call  center  operations. 
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written  communications  can  be  made  more  exacting,  and  beginning  to 
reliably  measure  both  customer  satisfaction  and  goal  achievement,  OWCP 
can  lay  the  foundation  for  better  serving  its  customers’  needs  for  timely 
and  accurate  information. 


Recommendations 
Executive  Action 


for 


We  recommend  that  the  Secretary  of  Labor  require  the 
to 


Director  of  OWCP 


•  establish  goals  for  all  important  areas  of  OWCP’s  telephone  and  written 
communications  with  iryured  workers  and  other  customers  and  revise  as 
appropriate  existing  goals  to  better  ensure  that  customers’  needs  for 
accurate  and  timely  information  are  met; 

•  collect  credible  performance  data  on  progress  toward  these  goals, 
including  timely  periodic  surveys  of  ii\jured  workers’,  medical  providers’, 
and  agencies’  satisfaction  with  OWCP’s  services  and  surveys  of  OWCP 
employees  to  gauge  their  job  satisfaction  and  to  gather  ideas  on  how  to 
improve  services;  and 

•  use  these  performance  data  and  survey  results  to  identify  areas  needing 
improvement  and  to  develop  strategies  for  achieving  those  improvements, 
including  new  and  revised  goals  where  appropriate. 


OWCP’s  Acting  Director  said  that  he  agreed  with  our  recommendations 
and  would  continue  to  explore  ways  in  which  to  improve  customer 
communications. 


That  concludes  my  statement,  Mr.  Chairman.  I  would  be  happy  to  answer 
any  questions  that  you  or  other  Members  of  the  Subcommittee  may  have. 
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To  determine  how  OWCP  communicates  with  injured  federal  workers, 
agencies  who  employed  these  workers,  and  medical  and  other  service 
providers  who  are  involved  in  their  treatment,  we  performed  the  following 
audit  steps. 

We  placed  2,400  telephone  calls  to  OWCP’s  12  district  offices  (200  per 
office),  to  assess  the  accessibility  of  telephone  representatives  at  each  of 
the  12  OWCP  district  offices  and  whether  each  office’s  automated 
integrated  voice  response  (IVR)  system’s  data  was  consistent  with  the 
national  office’s  claim  data.  We  generated  listings  for  our  callers  which 
provided  the  dates,  times,  and  phone  numbers  to  be  called,  and  when 
applicable,  the  claim  information  to  be  accessed.  OWCP  provided 
information  about  the  breakdown  of  IVR  call  types  for  four  of  the  twelve 
OWCP  district  offices  (Chicago,  Philadelphia,  Kansas  City,  and  Seattle) 
during  a  2-month  period.  We  used  this  information  to  build  a 
nongeneralizable  profile  of  the  distribution  of  IVR  calls  across  the 
following  six  types  of  claim  information  for  each  district  office  (number  of 
calls  per  district  office  in  parentheses): 

•  claimant  calling  about  a  bill  payment  (  47), 

•  medical  provider  calling  about  a  bill  payment  (8), 

•  medical  provider  calling  about  a  periodic  roll  payment  (24), 

•  claimant  calling  about  a  compensation  check  (10), 

•  claimant  calling  about  physical  therapy  authorization  (2),  and 

•  medical  provider  calling  about  physical  therapy  authorization  (9). 

For  the  IVR  calls,  we  were  provided  with  identifiers,  such  as  claim 
numbers  and  employer  identification  numbers,  that  enabled  us  to  enter  the 
IVR  and  identify  specific  transactions  on  each  office’s  automated  database. 
We  worked  with  OWCP  to  identify  a  time  period  of  earlier  transactions 
from  the  national  database  that  would  still  exist  in  each  district  office’s 
IVR  system  during  our  test  period  in  June  and  July  2000.  OWCP  sampled 
records  from  their  national  database  for  this  time  period  until  they 
obtained  the  required  number  of  records  of  each  type,  for  each  district 
office.  OWCP’s  selection  of  IVR  cases  did  not  strictly  constitute  a  random 
sample  of  cases  for  each  office  from  the  specified  time  period,  since  the 
cases  were  selected  in  case  number  order,  which  generally  reflected  a 
chronological  order,  until  the  required  number  of  cases  was  obtained. 
However,  since  we  had  no  reason  to  believe  that  time  ordering  of  the  cases 
was  associated  with  whether  a  district  office’s  IVR  data  would  match  the 
information  in  the  national  database,  we  accepted  OWCP’s  selections  for 
use  in  our  test. 
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All  of  our  test  calls  attempted  to  reach  an  OWCP  representative,  either 
directly  or  after  accessing  the  IVR  system.  We  therefore  randomly 
scheduled  all  our  test  calls  only  during  the  hours  that  each  office  told  us 
that  a  “live”  representative  should  be  available  to  respond  to  customer 
inquiries.  Saturdays,  Sundays,  and  Tuesday,  July  4,  2000,  were  excluded 
from  our  test  days. 


For  district  offices  that  had  the  same  telephone  number  for  the  automated 
voice  response  system  and  for  a  representative,  we  called  that  number  200 
times  and  attempted  to  reach  a  representative  immediately  upon  accessing 
the  system  on  100  of  those  calls  and  attempted  to  access  claim  information 
on  the  automated  system  and  then  access  a  representative  on  the  other  100 
calls.  For  offices  that  had  separate  numbers  for  the  automated  voice 
response  system  and  the  representative,  we  called  each  number  100  times. 
As  before,  upon  completion  of  the  automated  voice  response  system,  we 
attempted  to  reach  a  representative.  All  representatives  contacted  were 
informed  that  they  had  participated  in  a  GAO  telephone  survey.  When 
attempting  to  access  an  OWCP  office’s  telephone  system,  we  let  the 
telephone  ring  15  times  (over  1  minute)  before  making  the  determination 
that  the  telephone  system  did  not  answer.  And,  in  waiting  for  a 
representative  to  answer  the  line,  we  waited  at  least  5  minutes  after 
making  the  selection  to  speak  to  a  representative  before  determining  that 
the  call  was  not  answered.  We  used  the  5  minute  time  period  because  we 
wanted  to  be  more  conservative  than  VBA’s  goal  of  a  caller’s  accessing  a 
representative  within  3  minutes  and  NPR’s  guidance  and  Ohio’s  BWC’s 
goal  of  30  seconds.  For  each  call,  we  recorded  how  successful  the  district 
office  was  in  providing  services  an  injured  worker  or  medical  provider 
might  desire.  The  information  recorded  included  busy  signals,  no  answer 
after  1  minute,  whether  we  reached  an  office  representative,  whether  we 
reached  the  automated  voice  response  system,  and  whether  consistent 
information  was  provided  about  the  claimant  by  that  system,  such  as  the 
amount  of  a  medical  payment  or  the  status  of  medical  authorization  for 
physical  therapy.  We  conducted  this  telephone  survey  over  a  6-week 
period  in  June  and  July  2000,  and  attempted  the  calls  throughout  the 
business  hours  listed  for  each  district  office. 

If  the  test  calls  we  made  were  considered  random  samples  of  customers’ 
telephone  experiences  during  the  test  period,  the  following  statements 
could  be  made  about  the  precision  of  the  estimates: 

•  Estimates  of  the  proportion  of  calls  in  each  district  in  which  customers 
were  unable  to  access  the  telephone  systems,  and  the  proportion  of 
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calls  in  each  district  in  which  customers  were  unable  to  reach  an 
employee  within  five  minutes,  have  sampling  errors  of  no  more  than  10 
percentage  points. 

•  Estimates  of  the  proportion  of  IVR  calls  with  consistent  data  (among 
IVR  calls  for  which  transactions  could  be  tested)  have  sampling  errors 
of  no  more  than  10  percentage  points  unless  otherwise  noted  in  table 
II.1. 

The  data  we  collected,  however,  were  from  test  calls  rather  than  “actual” 
customer  calls.  Characteristics  of  the  test  calls  that  might  affect  the 
outcomes  we  measured,  such  as  time  of  day,  day  of  week,  or  subject 
matter,  might  not  have  mirrored  the  profile  of  these  characteristics  among 
“actual”  customers’  phone  calls  in  any  district  during  the  test  period. 
Therefore,  the  results  displayed  for  individual  districts  might  differ  from 
the  ones  we  might  have  obtained  by  sampling  “actual”  customer  calls,  by 
amounts  larger  than  the  stated  sampling  errors. 


•  At  OWCP’s  headquarters  in  Washington,  D.C.,  we  interviewed 

knowledgeable  officials,  reviewed  strategic  and  operational  plans  for  fiscal 
years  1995  through  2000  to  identify  goals  and  measures  related  to 
responding  to  customer  inquiries,  and  obtained  communication  reports 
showing  data  that  would  indicate  OWCP’s  performance  for  the  same 
period.  We  also  discussed  with  these  officials  methods  for  testing  the 
communications  at  OWCP  district  offices,  and  obtained  claimant  and 
medical  provider  information  so  that  we  could  perform  a  telephone  survey 
of  the  12  district  offices. 


•  We  visited  5  of  the  12  OWCP  district  offices — Chicago,  Dallas,  San 
Francisco,  Seattle,  and  Washington,  D.C.  At  these  five  offices,  we 
conducted  an  in-depth  review;  we  interviewed  regional  directors,  district 
directors,  claims  managers,  claims  examiners,  and  workers  compensation 
assistants  to  gain  an  understanding  of  the  communications  at  OWCP  from 
various  perspectives.  We  obtained  available  communication  reports, 
accountability  review  reports,  reports  measuring  written  and  telephone 
communication  performance,  and  other  applicable  communication  data. 
We  selected  the  five  offices  based  on  several  factors  to  obtain  a  mixture  of 
offices  of  differing  sizes  and  levels  of  performance.  The  primary  factors 
were  (1)  the  number  of  employees  and  the  number  of  cases  managed  at 
these  offices,  which  ranged  from  among  the  lowest  to  among  the  highest  of 
all  the  district  offices;  (2)  OWCP’s  telephone  and  written  responsiveness 
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measures,  which  indicated  that  some  offices  had  and  some  had  not  met 
national  performance  goals;  and  (3)  the  proximity  of  two  of  the  offices  to 
our  Washington,  D.C.,  and  Dallas  Regional  Office  staff. 


•  We  surveyed  the  other  seven  district  offices-Boston,  Cleveland,  Denver, 
Jacksonville,  Kansas  City,  New  York,  and  Philadelphia--via  a  questionnaire 
and  obtained  general  information  about  their  communication  practices, 
such  as  the  number  of  employees  each  assigned  to  respond  to  telephone 
calls. 

•  To  compare  OWCP’s  goals  and  practices  for  telephone  communication 
with  those  of  leading  organizations,  we  also  surveyed  three  agencies  that 
have  won  awards  for  their  telephone  communication  practices:  the  Social 
Security  Administration,  Department  of  Veterans  Affairs’  Bureau  of 
Benefits  Administration,  and  state  of  Ohio’s  Bureau  of  Workers’ 
Compensation  (BWC).  We  asked  OWCP  and  the  three  agencies  to  identify 
which  of  95  telephone  “best  practices”  they  used  that  NPR  identified  in  a 
1995  study.  We  did  not  attempt  to  verify  or  validate  their  responses.  In 
addition,  we  interviewed  and  obtained  documents  on  communication 
performance  goals  and  practices  from  officials  at  the  three  organizations 
and  compared  OWCP’s  performance  goals  and  measures  with  those  of 
private  sector  organizations  identified  in  the  NPR  study  and  those  of  the 
three  organizations  we  surveyed. 


We  did  our  work  between  January  and  September  2000  in  accordance  with 
generally  accepted  government  auditing  standards. 
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Access  Rates  For  And  Accuracy  Of  OWCP 
District  Offices’  Automated  Interactive  Voice 
Response  Systems 


Table  11.1:  Number  of  GAO  Survey  Telephone  Calls  Able  to  Access  and  Complete  OWCP  Interactive  Voice  Response  (IVR) 
Systems,  Number  of  Transactions  Tested,  and  Number  of  Transactions  With  Consistent  Claims  Data  by  District  Office 


District  office 

Number  of  calls 

made  to  attempt  Number  of  calls  able  Number  of  calls  able  Number  of 

access  to  IVR  to  access  IVR  to  complete  IVR  transactions  tested8 

Number 
(percentage)  of 
transactions  with 
consistent  datab 

Boston' 

100 

100 

98 

59 

55(93) 

Chicago' 

100 

79 

77 

51 

48(94) 

Cleveland' 

100 

93 

91 

54 

49(91  )d 

Dallas' 

100 

95 

95 

51 

48(94) 

Denver 

100 

96 

96 

60 

56(93) 

Jacksonville' 

100 

84 

63 

32 

28(88)e 

Kansas  City 

100 

93 

93 

59 

58(98) 

New  York' 

100 

90 

90 

52 

47(90)* 

Philadelphia 

100 

95 

81 

46 

44(96) 

San  Francisco 

100 

63 

63 

31 

31(100)" 

Seattle 

100 

99 

97 

59 

55(93) 

Washington  DC 

100 

75 

75 

42 

42(100) 

Totals' 

1200 

1062 

1019 

596 

561(94) 

Although  we  made  100  calls  to  each  district,  the  number  of  transactions  tested  at  each  district  are 
less  than  100  because  either  we  could  not  access  the  automated  voice  response  system  (e.g.,  busy 
signals,  no  answer,  etc.),  could  not  complete  the  transaction  (e.g.,  lost  connection,  system  rejected 
claim  input,  etc.),  or  the  system  had  been  updated  by  more  recent  data,  (i.e.,  information  was  updated 
after  OWCP  provided  the  sample  claims). 

b  We  compared  data  obtained  from  automated  voice  response  system  with  claim  data  provided  by  the 
national  office  and  determined  whether  each  district  office’s  system’s  data  were  consistent  with  the 
national  office’s  claim  data.  We  did  not  independently  verify  the  accuracy  of  the  national  office's  data. 

cThe  district  office  had  separate  telephone  numbers  for  accessing  the  automated  interactive  voice 
response  system  and  accessing  an  OWCP  representative.  The  other  district  offices  had  only  one 
main  number. 

d  If  the  test  calls  were  considered  as  a  random  sample,  the  lower  bound  of  the  sampling  error 
associated  with  this  estimate  is  1 1  percentage  points  less  than  the  value  of  the  estimate. 

B  If  the  test  calls  were  considered  as  a  random  sample,  the  lower  bound  of  the  sampling  error 
associated  with  this  estimate  is  16  percentage  points  less  than  the  value  of  the  estimate. 

f  An  average  for  all  OWCP  district  offices  cannot  be  provided,  because  OWCP  could  not  provide  the 
total  number  of  telephone  calls  each  office  received,  and  thus  we  could  not  weight  the  sample  to 
accurately  reflect  the  impact  of  each  district  office’s  performance  on  a  national  average. 

Source:  GAO  analysis  of  telephone  survey. 
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Viewing  GAO  Reports  on  the  Internet  . 

Pqr  information  on  how  to  access  GAO  reports  on  the  INTERNET, 
sentke-mail  message  with  “info”  in  the  body  tw 


info@\ 


i-gao.gov 


or  visit  GAO’&'World  Wide  Web  Home  Bage  at: 

http://www.gao.goVv  / 

Reporting  Fraud.  WastiK.and  Abuse  in  Federal  Programs 

To  contact  GAO's  Fraud  H(wftne  use: 

Web  site:  http://www.gajKgov/fraildnet/fraudnet.htm 
E-Mail:  fraudnet@gao^ov  n. 

Telephone:  1-800-424-5454  ( automatecbans  wering  system) 
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